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ABSTRACT 
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the  specified  military  reliability  requirements.  This  report  discusses  and 
details  those  factors  which  must  be  developed  by  prospective  contractors 
to  assure  responsiveness  to  the  reliability/quality  aspects.  Although 
the  emphasis  is  on  electronic  systems  and  equipments,  the  basic  precepts 
described  are  readily  adaptable  to  all  Air  Force  systems  and  equipments. 
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SECTION  1 
INTRODUCTION 


Design  of  reliable  electrical  and  electronic  circuits  and  equip¬ 
ment  has  shifted  from  corrective  philosophy  and  techniques  to  preventive 
philosophy  and  techniques.  In  the  past,  design  for  mission  performance 
came  first;  reliability  design  was  a  modification  kit  produced  only  after 
field  failures  made  a  more  reliable  design  imperative.  Today  the  emphasis 
is  on  increased  inherent  reliability  in  the  early  and  initial  design  stage. 
This  emphasis  can  be  readily  understood  when  one  considers  that  current 
electronic  systems  can  have  part  populations  totaling  upwards  to  1^50,000,000. 
Compare  this  with  World  War  II  Radars,  complex  in  their  time,  that  rarely 
exceeded  i4.5>000  parts.  Or  further,  compare  this  to  missile  systems,  whose 
piece  parts  are  in  the  order  of  50,000. 

Many  of  today's  ground  electronic  systems  are  one-of-a-kind,  and 
production  in  quantities  is  the  exception  rather  than  the  rule.  In  some 
instances,  major  items  such  as  radars  and  communication  equipments,  are 
procured  in  quantities;  however,  a  "second  chance"  to  redesign  an  elec¬ 
tronic  system  for  improved  reliability  is  rare.  The  first  model  fabri¬ 
cated  is  usually  the  one  that  is  installed  in  the  field;  it  almost  always 
becomes  an  operational  item  of  the  military  inventory.  On  those  rare 
occasions  where  an  additional  model(s)  of  an  electronic  system  is  pro- 
c\ired,  the  second  resembles  the  first  in  performance  parameters  only. 

There  is  generally  a  dramatic  physical  and  electrical  difference  between 
the  first  and  subsequent  models  of  electronic  systems,  largely  due  to 
advances  in  circuit  designs,  components,  and  solid  state  techniques, 
that  occurred  after  the  first  model  was  designed. 

This  report  concentrates  on  "ground"  electronic  systems;  however, 
the  approach  to  reliability  requirements  for  other  types  of  Air  Force 
electronic  systems  and  equipments  could  be  similiar.  The  scope,  depth 
and  extent  of  a  reliability  program  varies  with,  and  must  be  tailored  to, 
the  specific  procurement  situation. 
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SECTION  II 


BACKGROUND 


Basic  Air  Force  Systems  Command  Approach  and  Policies; 


The  increased  importance  of  Reliability,  particularly  in  the  pre¬ 
contract  and  contract  negotiation  stages,  is  emphasized  in  the  revised 
AFSCR  80-1,  "Reliability  Program  for  Systems,  Subsystems,  and  Equipments." 
This  regulation  can  be  summarized  in  terms  of  its  effect  on  Technical 
Proposals  as  follows: 

a.  "Quantitative  reliability  requirements  will  be  stated  in  spe¬ 
cific  numerical  terms  in  appropriate  contractual  documents  and  systems 
specifications  and  will  include  among  other  things,  the  confidence  levels 
to  which  the  specified  probabilities  will  be  demonstrated." 

b.  "Contracts  for  systems,  subsystems,  and  equipment  will  include 
a  requirement  for  a  comprehensive  and  organized  contractor  reliability 
program  that  extends  through  subcontractor  and  vendor  levels." 

c.  "Integrated  test  plans  to  investigate  causes,  effects,  modes 
of  failures,  and  to  demonstrate  achieved  reliability  will  be  developed  and 
incorporated  in  contractual  documents." 

d.  "New  programs  will  include  adequate  funds  for  reliability 
effort  in  design  proposals  and  initial  program  funding." 

e.  "Proposals  for  increasing  reliability  effort  and  funds  during 
a  program  will  be  screened  to  assure  that  the  effort  is  over  and  above 
that  already  required.  Decisions  to  accept  or  reject  such  proposals  will 
be  based  on  their  effect  on  spares,  maintenance  workload,  and  operating 
costs. " 

Reliability  and  the  System  Project  Offices 

AFSC  Regulation  80-1  requires  System  and  Project  Office  to  have 
the  necessary  technical  talent  available  to  monitor  and  assure  the 
reliability  aspects  of  system  and  equipment  contracts.  At  this  point 
in  time,  two  years  since  this  regulation  was  issued,  there  has  been  a 
dramatic  change  in  Air  Force  System  and  Project  Offices  concerning  Re¬ 
liability. 

AFSCR  80-lA  initiated  the  following  important  changes: 

a.  Each  AFSC  Division  is  now  required  to  have  an  active,  pro¬ 
fessional  Reliability  Staff.  This  Staff  insures  that  all  Development 
Plans,  System  Package  Programs,  Specifications,  Work  Statements,  Purchase 
Request  Packages,  etc.,  contain  reliability  requirements,  as  pertinent. 
Further,  the  Division  Staffs  review  overall  progress  and  compliance  to 
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reliability. 

b.  Equally  important,  the  individual  System  and  Project  Offices 
must  have  active,  qualified  Reliability  Monitors,  on  a  full-time  basis. 

Perhaps  the  most  interesting  event,  conceived  and  initiated  by 
the  Hq  AFSC  Reliability  Staff,  was  a  program  established  by  the  Air  Force 
Institute  of  Technology  leading  to  a  Masters  Degree  in  Reliability  Engin¬ 
eering.  The  candidates  for  this  program  are  carefully  chosen  from  highly 
qualified  engineering  Air  Force  officers  and  civilians.  Upon  completion, 
the  graduates  are  assigned  to  System  Project  Offices  as  full-time  Reliab¬ 
ility  Monitors.  The  System  and  Project  Offices  are  thus  being  staffed  with 
qualified,  competent,  professional  reliability  engineers;  engineers  who 
can  competently  challenge  as  well  as  evaluate  contractor's  reliability 
efforts.  In  some  respects,  academically  speaking,  the  Air  Force  is  acquir¬ 
ing  a  higher  level  of  reliability  talent  than  is  generally  available  to 
Industry.  It  is  predicted  that  both  industry  and  the  Air  Force  will  ben¬ 
efit  from  the  Masters  Program  in  Reliability. 

Reliability  -  A  Technical  Evaluation  Factor 


The  majority  of  Research,  Development,  Service  Test,  and  Pro¬ 
duction  procurements  require  that  Reliability  be  evaluated  as  one  of  the 
major  technical  factors  during  the  source  selection  and  contract  negotia¬ 
tion  phase . 

Most  of  today's  technical  proposal  evaluations  assign  a  percent¬ 
age  or  weight  factor  to  the  adequacy  of  the  Reliability  approach;  therefore, 
it  is  possible  to  gain  or  lose  a  number  of  points,  depending  on  the  accept¬ 
ability  of  the  submitted  data.  It  is  not  advisable  to  quote  a  weight- 
norm  generally  assigned  to  reliability  as  it  varies  for  technical  proposals. 
Reliability  may  be  "weighted"  from  two  to  twenty  per  cent,  depending  on 
the  procurement  situation,  the  type  of  equipment  being  procured,  the  in¬ 
tended  operational  use,  etc. 

Reliability  scoring  includes  "design  for  maintainability",  "sim¬ 
plicity  of  hardware/equipment  design",  and  the  "compatibility  of  the 
quality  control  program."  Reliability,  therefore,  is  one  of  the  primary 
factors  of  the  proposal  evaluation  criteria.  A  technical  Proposal  with 
an  xmsatisf actory  reliability  content  could  readily  render  the  entire 
proposal  unacceptable. 

Technical  Proposal  Evaluation  Boards 


The  cost  of  preparing  a  proposal,  particularly  those  prepared 
in  response  to  an  official  Request  for  Proposal  is  important.  Many  thou¬ 
sands  of  dollars,  and  sometimes  tens  of  thousands,  are  expended  in  the 
development  of  a  proposal.  Therefore,  proposals  should  be  more  productive, 
to  both  industry  and  the  military. 

The  Air  Force  evaluates  proposals  with  boards  consisting  of 
qualified  professional  engineers  and  technical  personnel.  Virtually 
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every  Board  has  a  voting  reliability  engineer,  whose  prime  mission  is  to 
determine  ii*  the  proposal  has  or  has  not  met  the  reliability  requirements. 
This  same  individual  is  frequently  associated  with  the  subsequent  relia¬ 
bility  activities  during  the  life  of  the  contract. 

One  of  the  responsibilities  of  the  Evaluation  Boards,  not  gen¬ 
erally  known,  is  to  determine  if  certain  portions  of  proposals  not  selected 
are  of  interest  from  a  technique  aspect.  The  reliability  representative, 
for  example,  may  discover  a  new  or  unusual  approach  to  a  current  relia¬ 
bility  problem  that  might  be  of  interest  to  another  program  or  project,  In 
this  situation,  the  proposer  would  be  encouraged  to  bid  separately  for  just 
that  portion  of  the  proposal  that  is  of  interest,  or,  the  company  would  be 
encouraged  to  submit  a  separate  proposal,  which  would  then  be  treated  on 
a  sole-source  basis.  In  this  area  particularly,  there  is  a  wide  use  of 
not-for-profit  organizations  in  the  evaluation  process. 

Not-for-profit  organizations  are  used  in  the  technical  direction 
and  overall  general  systems  engineering  of  ground  electronic  projects  and 
programs.  The  not-for-profits  are  also  now  being  utilized  as  advisors 
to  Evaluation  Boards.  They  participate  in  the  evaluation  of  all  portions 
of  the  proposal  except  the  cost,  which  is  never  made  available  to  them. 

Evaluation  Boards  today  are  a  direct  answer  to  industry's  con¬ 
cern,  that  when  good  technical  proposals  have  been  prepared  by  industiy, 
fully  qualified  Air  Force  technical  personnel  be  utilized  to  evaluate 
these  proposals. 

New  Air  Force  System  Life  Cycle 

The  new  Air  Force  Systems  Command  Manual  AFSCM  375-U,  "System 
Program  Management  Manual",  describes  the  activities  and  events  that  all 
new  Air  Force  systems  will  be  required  to  establish  and  to  be  subject  to 
during  the  life  cycle  of  the  system.  Briefly,  the  new  system  life  cycle 
is  summarized  in  Figure  No.  1  (page  6  ). 

The  following  are  the  reliability  events  and  activities  that 

occur: 

a.  Conceptual  Phase.  The  primary  objective  during  the  Concep¬ 
tual  Phase  is  to  evolve  and  delineate  realistic  Reliability  objectives 
that  will  satisfy  the  operational  mission.  This  is  the  first  opportimity 
to  identify  the  features  which  affect  the  reliability  characteristics. 

A  study  and  investigation  is  made  of  the  reliability  aspects,  and  the 
implications  of  trade-offs  between  reliability  and  other  constraints. 
Realistic  reliability  goals  are  fully  evolved. 

b.  Definition  Phase.  Detailed  quantitative  reliability  re¬ 
quirements  are  established  during  this  Phase  and  are  integrated  in  the 
detailed  System  Design  Specification,  which  is  the  primary  product  of  the 
Definition  Phase.  Reliability  requirements  are  allocated  to  lower  func¬ 
tional  levels,  and  the  impact  on  maintenance  and  other  associated  functions 
defined.  In  essence,  a  preliminary  Reliability  Program  Plan  is  established 
at  this  point  in  time. 
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Figure  1.  New  Air  Force  System  Life  Cycle 
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c.  Acquisition  Phase.  During  this  phase  the  contractor  implements 
the  reliability  objectives,  and  the  reliability  program  which  was  detailed 
and  defined  during  the  Definition  Phase.  The  contractor  also  updates  and 
revises  the  Reliability  Program  as  necessary  to  meet  the  contractual  quan¬ 
titative  reliability  requirements.  The  engineering  inspections  and  test 
programs  are  conducted  at  this  time  to  determine  the  compliance  of  relia¬ 
bility  with  contractual  requirements. 

The  principal  significance  to  Reliability  in  the  new  System  Life  Cycle, 
is  the  opportunity  to  adequately  insure  that  Reliability  is  initially  con¬ 
sidered,  that  realistic  reliability  requirements  are  established,  and  that 
reliability  is  in  fact  reflected  in  the  delivered  end  items. 
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SECTION  III 


ANALYSIS  OF  RELIABILITY  REQUI REMENTS 


Approach 


The  first  objective  when  analyzing  the  reliability  requirements 
of  a  Request  for  Proposal  (RFP)  Package  is  to  insure  that  the  stated  re¬ 
quirements  and  terminology  are  mutually  understood,  since  eventually  they 
must  be  agreed  to  by  the  customer  and  the  potential  contractor.  However, 
in  actual  practice  detailed  reliability  requirements  are  sometimes  lacking, 
for  the  valid  reason  that  the  manner  of  accomplishment  of  the  mission  is  not 
firm. 


When  Reliability  has  not  been  quantitatively  specified  by  the 
Air  Force,  the  reliability  objectives  and  the  reliability  program  consis¬ 
tent  with  product  use  must  be  established.  All  requirements  should  be  re¬ 
lated  to  operational  figures  of  merit  such  as  "percent  up  time,"  "mission 
success  probability,"  "in-commission  rates,"  or  "average  maintenance  times." 
A  contractor  then  can  establish  realistic  levels  derived  from  mathematical 
models  of  system  and  weapon  effectiveness,  and  based  on  a  study  of  the 
functions  of  specific  system  mission  requirements.  Overall  system  require¬ 
ments  are  then  "budgeted"  for  each  system,  subsystem,  and  major  component. 

In  general,  the  Air  Force  states  reliability  quantitatively  when  designs 
are  firm.  Conversely,  reliability  is  stated  qualitatively  where  only  goals 
are  known  and  have  been  given. 

The  work  statement  on  reliability  states  the  function  the  pro¬ 
posed  equipment  performs  in  operational  use,  and  is  further,  an  inter¬ 
pretation  of  reliability  as  it  applies  to  broad  performance  parameters. 

A  prospective  contractor  must  define  this  function  and  his  inter¬ 
pretation  of  quantitative  reliability  as  he  understands  it.  Quite  often 
the  interpretation  may  vary  depending  on  the  configuration  proposed.  This 
will  then  clarify  exactly  what  product  reliability  the  prospective  con¬ 
tractor  is  quoting. 

Contractor  proposals  must  establish  reliability  in  proper  per¬ 
spective;  that  is,  a  proposal  must  show  that  the  bidder  is  aware  of  the 
relationships  between  reliability,  maintainability,  logistics,  human 
engineering,  design  simplicity,  etc.  The  proposal  must  not  isolate  re¬ 
liability  as  a  separate  science,  but  must  show  the  relationship  to,  and 
trade-offs  with,  other  functions  and  their  design  interactions.  Often 
the  best  approach  to  a  difficult  reliability  problem  is  a  design  ensuring 
high  maintainability,  particularly  where  the  state-of-the-art  in  reliability 
will  not  in  itself  result  in  the  reliability  level  required  for  a  specific 
procurement.  An  approach  that  is  particularly  adaptable  to  electronic 
systems  is  that  of  " availability" . 
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Availability 


Availability  is  the  proper  balance  and  application  of  relia¬ 
bility  and  maintainability  techniques.  Two  factors  are  necessary  to 
calculate  inherent  availability;  mean  time  before  failure  and  mean  time 
to  restore  service  (in  an  excellent  environment).  The  mean  time  before 
failure  is  a  reliability  factor,  and  the  trouble  clearing  time  is  a 
maintainability  factor.  Low  orders  of  reliability  can  frequently  be 
greatly  improved  by  a  corresponding  improvement  in  maintainability. 
Availability  improves  as  the  maintenance -action  rate  improves. 

The  militaiy  commander  in  the  field  is  more  confident  knowing 
that  there  is  a  probability  that  his  equipment  will  be  available  to 
perform  a  mission,  than  he  is  knowing  that  the  Mean  Time  Between  Failure 
(MTBF)  is  a  high  absolute  number.  Further,  the  military  commander  wants 
to  be  assured  that  he  can  initiate  and  successfully  execute  any  particu¬ 
lar  mission,  and  this  presupposes  having  the  equipment  available  at  the 
time  of  the  mission. 

In  its  basic  form,  the  following  relation  is  given  to  define 
availability: 

MTBF 

Availability  =  _ 

OTBF  +  MTTR 

The  highert  value  A  can  have  is  iinity,  or  100  per  cent.  Avail¬ 
ability  is  a  ratio  of  system  operating  hours  to  the  sum  of  system  oper¬ 
ating  plus  maintenance  hours. 

There  are  two  methods  to  approach  100  per  cent  Availability  of 
an  electronic  system.  The  system  can  be  built  (for  a  given  mission  time) 
to  have  such  a  high  MTBF  in  relation  to  MTTR  that  Availability  will  be 
practically  100  per  cent,  or  the  ease  of  maintenance  can  be  designed 
into  the  equipment  to  shorten  the  MTTR  so  much  that  this  will  also  result 
in  a  very  high  Availability  and  permit  the  system  to  perform  its  mission, 
even  with  modest  MTBF.  The  large  number  of  serial  parts  of  many  elec¬ 
tronic  system  prohibits  the  design  for  an  order  of  reliability  which  will 
obtain  the  required  order  of  high  Availability  through  reliability  tech¬ 
niques  alone.  The  design  for  efficient  and  rapid  maintainability  is 
frequently  the  most  practical  way  to  supplement  the  design-for-reliability 
and  achieve  the  required  level  of  Availability,  since  attempts  to  incor¬ 
porate  more  and  more  reliability  generally  skyrockets  the  cost  of  equipment. 

It  is  interesting  to  note  that  the  Martin  Marietta  Handbook, 
"Estimating  Reliability,"  defines  Availability  as  "Efficiency."  A 
recent  Navy  paper  delivered  in  Boston  described  Availability  as  "Depen¬ 
dability,"  and  utilized  reliability  and  maintainability  as  the  prime 
components  of  the  dependability  equation. 

In  general,  the  submission  of  separate  plans,  one  for  main- 
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tainability  and  one  for  reliability,  is  discouraged.  This  is  due  to  the 
und.que  operational  requirements  of  electronic  systems.  These  requirements 
involve  a  numerical  expression  for  operational  availability.  The  latter 
represents  a  simultaneous  treatment  of  maintainability,  expressed  as  a 
me an- down- time  (MDT)  statistic;  and  reliability,  expressed  as  a  mean-time- 
between-failure  (MTBF)  statistic. 

Technical  Proposals  for  electronic  systems  are  therefore  re¬ 
quired  to  include  one  combined  program  plan.  A  plan  which  will  describe 
the  methods  by  which  a  contractor  details  the  control  of  both  the  MDT 
and  the  MTBF  characteristics  of  electronic  systems,  beginning  with  the 
design  phase  of  a  system  program.  This  plan  can  be  referred  to  as  an 
Availability  Program  Plan.  In  substance  then  Availability  gives  equipment 
designers  an  area  for  reliability/maintainability  trade-offs  based  on 
achieving  a  required  availability  at  minimum  design  production,  and  main¬ 
tenance  costs. 

Initial  Prediction  of  Reliability 


Technical  Proposals  for  electronic  systems  must  include  a  pre¬ 
liminary  analysis  of  reliability  requirements.  The  bidder  is  required  to 
make  a  "first  prediction"  of  reliability  capability  for  the  designs/ 
equipments  that  he  is  proposing.  The  bidder  must  then  discuss  and  compare 
the  results  of  his  "first  prediction,"  with  the  desired  reliability  re¬ 
quirements.  It  is  never  sufficient  for  the  contractor  to  merely  state 
that  he  "will  meet  all  required  reliability  requirements,"  or  that  he  "will 
comply  with  all  contractual  requirements."  It  is  important  that  all 
derived  equations  and  mathematical  assumptions  are  clearly  stated,  so 
that  they  may  be  verified  by  the  members  of  the  Technical  Proposal  Review 
Board.  The  proposer  is  required  to  make  an  estimate  of  the  reliability 
which  can  be  achieved  during  the  course  of  the  contract,  and  to  provide 
data  which  will  validate  this  e  stimate . 

While  developing  the  reliability  estimation,  preliminary  quan¬ 
titative  numbers  are  developed  by  the  bidder  which  will  be  based  on 
existing  or  known  failure  rate  data  sources.  It  is  necessary  at  this 
point  to  detail  the  failure  rate  data  sources  and  to  discuss  the  adequacy 
of  the  sources  utilized.  The  prospective  contractor  must  indicate  the 
degree  of  confidence  and/or  experience  that  he  has  had,  with  the  accuracy 
of  the  failure  rate  data  sources  that  he  utilized  for  his  "first  pre¬ 
diction.  " 


A  primary  objective  of  early  reliability  prediction  is  iden¬ 
tification  of  any  possible  reliability  weaknesses  in  the  proposed  design. 
At  this  point  in  time,  knowing  how  any  possible  reliability  design  weak¬ 
nesses  are  to  be  overcome  is  of  interest.  This  is  the  proposer's  oppor¬ 
tunity  to  detail  and  discuss  trie  compensating  features  that  are  to  be 
utilized.  As  an  example,  the  state-of-the-art  may  indicate  that  redun¬ 
dancy  of  functions  may  have  to  be  used  in  order  to  meet  the  estimated 
reliability. 
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The  maximum  environmental  and  stress  conditions  to  which  the 
system  may  be  subjected  during  its  various  liie  phases  must  be  estimated. 
The  proposer  is  required  to  identify  the  anticipated  problem  areas 
relative  to  the  capability  of  the  components,  equipments,  subsystems, 
etc.,  to  withstand  the  estimated  environmental  or  stress  conditions  and 
manufacturing  limitations. 

VJhenever  Government  Furnished  Equipment  or  sub-contractors' 
equipment  is  to  be  integrated  into  the  system,  the  proposer  must  show  the 
compatibility  of  this  equipment  with  the  desired  reliability  requirement. 
The  proposer  will  identify  the  reliability  problems  which  have  been 
introduced  by  equipment  over  which  the  contractor  has  no  control;  for 
example.  Government  Furnished  Equipment.  The  proposer  must  indicate 
the  degree  of  improvement  necessary  to  make  this  equipment  consonant 
with  the  rest  of  the  equipment  comprising  the  total  system  or  subsystem. 

Finally,  the  "first  prediction"  will  acknowledge  as  well  as 
identify  the  proposed  preventive  maintenance  cycle  that  will  be  required 
to  maintain  the  orders  and  levels  of  reliability  that  are  being  pre¬ 
dicted.  This  should  include  the  type  of  maintenance,  as  well  as  the 
duration  of  the  maintenance  cycle.  Discussion  must  clearly  identify 
the  extent  and  degree  of  preventive  maintenance  that  will  be  required. 

Design  Reviews 


Participation  in  the  review  and  evaluation  of  all  equipment 
and  component  design  is  considered  one  of  the  most  important  activities 
of  a  contractor's  reliability  department.  The  technical  proposal  is  ex¬ 
pected  to  clearly  indicate  the  specific  procedures  that  will  assure  that 
all  design  and  engineering  changes  occurring  during  the  entire  cycle 
(development  through  final  fabrication)  will  be  subject  to  effective 
reliability-design  reviews.  Further,  the  reviews  for  reliability  must 
not  be  of  an  "advisory"  nature.  The  reliability  department  must  be  in 
a  position  to  detect  and  eliminate  potential  causes  of  low  reliability 
and/or  low  maintainability  early  enough  in  the  program  to  re-design 
when  re-designs  are  the  least  costly.  However,  it  is  the  design  eng¬ 
ineer's  responsibility  to  effect  the  actual  re-design.  The  cost  in¬ 
volved  in  changes  during  the  initial  design  is  estimated  to  be  1/lOOOth 
that  of  the  cost  after  equipment  has  been  fabricated  and  delivered  to 
the  military  user  in  the  field.  Figure  2  is  a  graphical  illustration 
of  this  point,  that  it  costs  approximately  1000  times  more  for  re¬ 
designs  to  delivered  equipment  than  it  would  be  to  have  made  that  same 
change  in  the  Initial  Design  Phase.  Figure  2  was  developed  by  the  Air 
Force  Logistic  Command  and  was  detailed  in  a  paper  by  Lt  Col  Samuel 
H.  Lewis,  Wright-Patterson  AFB,  "Customer  Requirements  for  Maintain¬ 
ability  (M)  Characteristics  in  Air  Force  Systems  and  Equipment."  The 
ratios  for  the  costs  in  the  Production  Phase  and  the  Development  Phase 
are  also  shown. 
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Figure  2.  Initial  Design  Costs  vs  Re-Design  Costs 
Equipment  Reliability  Demonstration 


The  contractual  requirement  of  a  minimum  mean-time -be tween- 
failure  (MTBF)  for  an  equipment  introduces  the  problem  of  reliability- 
demonstration.  Electronic  systems  usually  are  operating  with  at  least 
four  constraints: 

a.  Equipment  MTBF's  are  in  the  order  of  500  hours  or  more. 

b.  Limited  numbers  (usually  one  or  two)  of  articles  are  purchased. 

c.  Delivery  schedules  are  compressed. 

d.  A  finite  number  of  dollars  are  available. 


With  such  constraints,  the  present  reliability  demonstration 
model  defined  in  USAF  Specification  MIL-R-26i;7U  requires  supplemental 
instructions  to  assure  a  satisfactory  solution  to  the  demonstration  prob¬ 
lem.  The  object  is  to  insure  that  no  suspended  decisions  will  be  necessary 
on  compliance  to  contractual  MTBF's.  These  can  last  as  much  as  a  year 
(many  electronic  equipment  programs  require  delivery  within  eight  to  ten 
months  from  contract  award).  A  quantitive  decision  rule  or  model  is  re¬ 
quired  which  has  at  least  the  following  essential  properties: 

a.  It  is  understood  by  both  the  customer  and  contractor. 

b.  Quantifies  the  risks  involved  in  decisions,  i.e.,  the 
probability  of  rejecting  equipments  which  have  achieved  the  contractual 
MTBF  and  the  probability  of  accepting  equipments  which  have  not  achieved 
the  contractual  MTBF. 

c.  Allows  scheduling  and  cost  analysis  within  the  overall  con¬ 
straints  of  the  program. 

A  simple  decision  rule  is  being  used  which  satisfies  the  above 
essential  properties.  This  rule  states  that  equipment  will  be  planned 
to  be  operated  for  a  specific  amount  of  time  (such  as  one  multiple  of 
contractual  MTBF)  and  diu’ing  this  time  a  specific  number  of  failures 
will  be  allowed.  If  this  number  is  exceeded,  the  equipment  will  be 
declared  unacceptable. 

For  example,  assume  the  case  where  test  time  is  set  equal  to 
Oq,  the  contractual  MTBF.  If  c  is  set  equal  to  zero,  a  contractor  would 
have  a  39%  chance  that,  even  if  his  equipment  has  an  MTBF  equal  to  29^, 
the  equipment  would  be  rejected.  On  the  other  hand,  the  customer  has  a 
2h%  chance  of  accepting  an  equipment  which  has  an  MTBF  equal  to  \/29q. 
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However,  to  minimize  the  contractors’  risk,  c  could  be  set  equal  to  one. 

If  the  equipment  had  an  MTBF  equal  to  IQc,  the  contractors'  risk  (pro¬ 
bability  of  rejection)  would  now  be  8%,  but  then  the  customer  would  have 
a  h2%  chance  of  accepting  an  equipment  which  has  an  MTBF  equal  to  1/2  Gg. 

This  basic  arithmetic  illustrates  the  need  for  clearly  defining 
and  obtaining  agreement  on  a  value  of  c.  It  is  important  that  some  dis¬ 
cussion  on  this  matter  take  place;  initially  in  technical  proposals  and 
later  in  contract  negotiations.  The  point  of  consideration  in  these  dis¬ 
cussions  is  to  make  both  parties  aware  of  each  others  risks. 

A  reliability  demonstration  can  be  through  equipment  testing  or 
through  analytical  or  mathematical  demonstration  coupled  with  monthly 
reports.  It  is  recognized  that  the  basic  sequential  model  presented  in 
Specification  MIL-R-26U7U  requires  additional  guidance  for  application  to 
a  complete  electronic  system  with  several  alternate  modes  and  redundant 
replacements  for  various  functional  circiiitry.  The  basic  sequential  model 
is  viewed  as  a  possible  vehicle  for  reliability  demonstrations  of  a  simple 
series  system,  a  particular  mode  of  operation  of  a  complex  system,  a  sub¬ 
system  of  a  system,  etc. 

It  is  necessary  that  in  the  discussion  of  equipment  reliability 
demonstration  plans  the  following  be  detailed  and  developed  in  the  Tech¬ 
nical  Proposal: 

a.  Prepare  a  general  plan  for  demonstration  of  the  achieved 
reliability  at  a  specified  point  in  time,  including  estimated  number  of 
test  articles,  and  confidence. 

b.  Develop  trade-off  curves  showing  number  of  test  articles  or 
cost  versus  confidence. 

c.  Detail  how  the  testing  for  reliability  at  the  system,  sub¬ 
system,  equipment  or  component  level,  separately  or  in  combination,  will 
be  performed. 

d.  State  conditions  and  groimd  rules  for  deciding  if  a  test 
shall  be  classified  as  a  success  or  failure,  or  shall  be  excluded  because 
of  invalid  test  data. 

e.  Indicate  the  scope  of  the  detailed  plans  for  conducting  a 
reliability  demonstration.  The  contractor  must  submit  these  plans  to  the 
customer  at  least  ten  days  prior  to  the  date  the  design  review  begins. 

f.  Detail  the  corrective  action  procedures  that  will  guarantee 
that  data  collection  results  and  design  reviews  are  effective. 

A  reliability  test  plan  is  not  considered  complete  until  a  clear 
indication  of  the  contractor's  failure  feedback  system  and  corrective 
loop  is  presented. 

Failure  Data  Collection 

Failure  rates  and  the  underlying  mathematical  model  are  uti¬ 
lized  for  the  important  area  of  spares  provisioning.  Current  costs  of 
spare  parts  for  Air  Force  electronic  equipments  and  systems  average  1$^ 
to  20%  of  the  total  contract.  Considering  the  total  costs  of  some  elec- 


tronic  systems,  spare  parts  alone  for  a  large  electronic  system  can  readily 
cost  millions  of  dollars.  Inadequate  Failure  Data  procedures  can  mean  a 
difference  in  millions  of  dollars  both  to  the  military  services  and  to 
the  taxpayers  by  excessive  spares  purchases.  Further,  it  is  possible  to 
increase  down  time  by  improper  spares  provisioning  and  thus  create  other 
types  of  costs  due  to  shortage  of  spare  parts. 

The  prospective  contractor  must  demonstrate  that  he  understands 
the  relationships,  the  interplay,  and  the  interdependency  between  Failure 
Data  Collection,  the  spares  provisioning  process,  the  maintenance  aspects, 
as  well  as  the  improvement  in  reliability  that  can  be  derived  by  es¬ 
tablishing  and  effecting  the  proper  Failure  Data  Collection  System. 

Specification  MIL-R-27^U2A  defines  "failure,"  however,  further 
information  for  application  to  electronic  system  Technical  Proposal 
Evaluations  is  usually  supplied.  It  is  important  that  "failure"  be  de¬ 
fined  by  the  contractor,  specifically  and  explicitly  for  the  particular 
equipment  or  system  under  procurement  and  as  the  contractor  intends  to 
apply  it  to  the  Reliability  Program.  Failure  in  one  circumstance  can  be 
a  simple  breakdown  of  a  component  such  as  a  receiver  or  transmitter. 

Failure  may  also  be  defined  as  the  inability  to  communicate  between  two 
geographical  locations.  Failure,  therefore,  can  be  through  equipment  or 
be  operational  or  even  a  combination  of  the  two.  Failure  of  a  single  com¬ 
ponent  can  be  catastrophic  when  the  component  is  the  only  one  of  its  type, 
or  is  one  in  a  chain  of  components.  On  the  other  hand,  a  failure  of  one 
of  the  sequential  parallel  transmitters  in  the  phased  array  radar  could 
not  be  considered  as  catastrophic  because  only  a  small  decrease  in  per¬ 
formance  would  result.  Failure  of  indicator  lamps,  test  points,  alarm 
circuitry,  etc.,  which  do  not  affect  system  performance,  are  not  normally 
included  in  the  MT3F  calculations.  The  technical  proposal  should  there¬ 
fore  indicate  an  understanding  of,  and  the  proper  application  of  the 
proposed  Failure  Data  Collection  System,  and  should  define  the  various 
categories  of  failure  as  they  are  used  in  the  contractor  technical 
proposal. 

Organization  for  Reliability 

There  are  many  ways  of  organizing  to  achieve  reliability.  It 
is  not  the  intent  here  to  describe  the  ideal  or  preferred  organization. 

The  bidder  must  develop  and  describe  an  organization  that  meets  the  re¬ 
quirements  of  a  particular  program.  A  Reliability  Group,  for  example, 
does  not  have  to  be  a  permanent  organizational  group;  it  is  satisfactory 
in  certain  instances  to  have  only  a  permanent  Chaiman,  with  members 
available  as  required.  The  following  aspects  are  considered  particularly 
important: 

a.  Qualified  Personnel:  It  is  necessary  for  the  prospective 
contractor  to  identify,  by  professional  skills,  the  personnel  to  be 
assigned  to  the  reliability  program.  An  assessment  will  then  be  made 
by  the  technical  evaluators  whether  the  number  (particularly  those  to 
be  assigned  full  time)  is  adequate,  considering  the  scope  of  the  pro¬ 
posed  overall  program.  Previous  experience  of  the  personnel  in  reliability 


design,  reliability  prediction  and  reliability  management  group  as  a 
whole,  in  the  areas  of  analysis,  design,  statistical  methods,  parts  and 
test  methods,  are  to  be  detailed,  and  are  of  particular  interest. 

The  end  product  under  procurement  to  a  large  degree,  dictates 
the  type  of  reliability  personnel  to  be  assigned  to  a  particular  contract. 
In  some  programs  personnel  with  general  reliability  engineering  backgrounds 
are  more  important.  If  extensive  reliability  mathematical  models  are  used, 
mathematicians  and  analysts  are  a  must,  and  a  lack  of  these  skills  might 
be  interpreted  as  a  lack  of  understanding  of  the  reliability  problems 
involved. 

b.  Reliability  Organization:  The  contractor's  organizational 
framework  is  of  interest  and  more  specifically,  the  structural  position 
of  reliability  activities  in  relation  to  engineering,  manufacturing,  and 
quality  assurance.  It  is  understood  that  not  everyone  can  or  even  should 
report  directly  to  the  president  of  the  corporation,  but  the  reliability 
organization  shoiild  be  in  a  position  to  participate  in  the  formulation  of 
design  criteria  and  to  effectively  control  the  design  of  the  end  item  for 
reliability.  It  is  particularly  important  that  the  Reliability  Group 
not  be  submerged,  in  manufacturing  for  example. 

c.  Effectiveness  of  Reliability  Organization;  It  is  necessary 
that  the  technical  proposal  demonstrate  that  the  prospective  contractor's 
Reliability  Group  will  be  truly  responsive  to  overall  end  item  requirements, 
sensitive  to  design  problem  areas  that  evolve  anywhere  in  the  company,  and 
that  organizationally  the  work  of  the  reliability  group  is  timely.  In 
other  words,  does  the  Reliability  Group  have  stature  and  authority?  Is 
there  a  positive  management  means  for  auditing  the  progress  of  the  re¬ 
liability  activities?  Is  there  an  effective  means  for  crosstalk  and 
feedback  of  information  between  the  reliability,  the  engineering,  manu¬ 
facturing,  test  and  quality  control  activities?  Conversely,  the  Re¬ 
liability  Group  should  not  have  an  overriding  authority  to  re-design 
equipment  or  circuits,  just  for  the  sake  of  reliability;  rather,  the 
contractor's  Design  Review  Teams  should  consist  of  an  active,  voting 
Reliability  representative,  who  will  participate  in  all  design  reviews, 

and  whose  considered  reliability  opinions  will  be  pertinent. 

d.  Utilization  of  Consultants  and  Advisors:  Listing  advisors 
and  consultants  on  an  "as  required"  availability  basis  is  a  very  common 
practice  among  bidders  today.  The  individuals  listed  are  generally 
nationally  recognized  specialists  or  authorities  in  their  particular 
fields.  Five,  ten,  or  e-ven  twenty  percent  of  their  time  may  be  promised 
to  the  contract,  but  frequently  the  individuals  appear  on  the  roster  only 
for  prestige  purposes.  Past  experience  has  shown  that  all  too  often 
these  "big  names"  are  not  assigned  to  the  program  or  spend  insufficient 
time  to  be  of  tangible  benefit  to  the  program.  The  Technical  Proposal 
must  indicate  the  specific  functional  responsibilities  or  areas  to  which 
the  consultants  and  advisors  will  be  assigned,  and  the  specific  con¬ 
tributions  that  they  will  be  expected  to  make  to  the  total  program. 

Another  practice  common  today  shows  Vice  Presidents,  Depart¬ 
ment  Managers,  or  even  Presidents,  as  overall  project  leaders.  Again, 
the  technical  proposals  must  carefully  specify  the  precise  function 
that  executives  will  perform,  since  it  is  known  that  in  practice,  it  is 
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primarily  the  full-time  project  personnel  that  can  be  depended  upon  for 
the  day-to-day  continuity  required  to  bring  any  reliability  program  to  a 
successful  completion.  Vice  Presidents  and  Chief  Engineers  can  rarely 
devote  sufficient  time  to  project  engineering  without  sacrificing  their 
demanding  and  time-consuming  executive  responsibilities. 

Consultants  and  advisors  have  made  valuable  contributions  to 
many  defense  contracts.  Their  participation  is  encouraged,  if  there  is 
a  genuine  need  for  their  talents  and  experience,  and  provided  that  a 
consultant's  commitment  of  time  is  made  in  writing,  and  is  submitted  as 
part  of  the  Technical  Proposal  package. 

Reliability  Incentives/Penalties 


A  number  of  recent  groimd  electronic  system  contracts  have  had 
the  reliability  effort  priced  separately.  This  has  caused  increased 
attention  to  the  reliability  aspects,  however,  this  has  also  led  to 
additional  Air  Force  controls  over  that  portion  of  the  contract  costs. 

One  of  the  resultant  controls  is  the  Reliability  Incentives  and  Relia¬ 
bility  Penalties. 

The  Air  Force  and  the  Department  of  Defense  have  expressed  a 
growing  concern  about  inadequte  contractual  provisions  for  the  assur¬ 
ance  of  system/equipment  reliability.  As  a  result.  Cost  Plus  Fixed 
Fee  (CPFF)  contracts  are  being  used  on  a  highly  selected  basis  only, 
and  are  being  replaced  by  incentive  fee  type  of  contracts.  Contracts 
with  established  incentive  fees  have  been  expressed  quantitatively  in 
mean-time-between-failures  (MTBF)  and  a  reliability  program  tailored 
to  the  general  requirements  of  MIL-R-275^2  has  been  established.  The 
primary  objective  of  the  incentive  fee  is  to  preclude,  or  at  least 
minimize,  the  various  uncontrollable  factors  which  arise  during  the 
course  of  a  contract,  and  result  in  failure  to  meet  the  reliability 
requirements. 

The  inclusion  of  reliability  incentive  fees  is  intended  to 
insure  that  contractor-allocated  funds  for  the  contractor's  reliability 
program  will  not  be  redirected  to  some  other  contract  "activity,  partic¬ 
ularly  when  the  contractor  is  experiencing  some  difficulty  with  other 
facets  of  the  contract.  Much  of  the  DOD  guidance  on  Incentive  Fee 
Contracts  applies  directly  to  the  reliability  portions,  and  is  partic¬ 
ularly  applicable  to  electronic  systems. 

Experience  with  electronic  systems  has  indicated  that  the 
system  hardware  evolved  during  the  R&B  effort  usually  becomes  the 
system  that  is  delivered  and  installed  as  an  operational  system.  This, 
therefore,  dictates  that  the  initial  design  must  be  the  right  one,  since 
there  is  no  second  chance  to  significantly  increase  low  or  unsatisfactory 
orders  of  reliability.  The  DOD  Guide  on  Incentive  Fees  states  that  "a 
contractor  should  be  motivated,  in  calculable  monetary  terms  to: 

a.  Turn  out  a  product  that  meets  significantly  advanced  per¬ 
formance  goals. 

b.  To  improve  on  the  contract  schedule  up  to  and  including 
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final  delivery. 

c.  To  substantially  reduce  the  costs  of  the  xrork,  or, 

d.  To  complete  the  project  under  a  weighted  combination  of  some 
or  all  of  these  objectives" 

The  DOD  instructions  are  quick  to  caution,  however,  that  the 
performance  incentive  (including  reliability)  will  not  be  used  under  any 
circumstance,  as  a  substitute  for  a  clear  definition  of  the  desired  end 
item.  It  is  intended  that  the  performance  incentive  emerges,  not  as  a 
means  of  turning  the  program  into  a  profit  "game"  operating  between  very 
wide  limits,  but  simply  as  an  inducement  to  the  contractor  to  meet  or 
exceed  the  nominal  performance  requirements  as  established  in  the  contract. 

In  a  recent  electronic  communications  system  contract  award, 
the  following  provisions  regarding  reliability  were  instituted: 

a.  The  contract  is  a  cost-plus -incentive  fee  (CPIF)  type,  with 
performance,  cost,  and  the  delivery  as  incentive  features. 

b.  The  overall  incentive  arrangements  are  weighed  as  follows: 
Performance  -  60%,  Delivery  -  10^,  Cost  -  30%. 

c.  Under  the  Performance  weights.  Reliability  is  weighed  at 
2k%,  and  is  based  on  the  amount  by  which  the  minimum  specified  reliability 
and  confidence,  as  stated  in  the  Statement  of  Work,  are  exceeded  by  com¬ 
pletion  of  ground  testing. 

Figure  3  pictorially  depicts  a  progressive  relationship  for 
reliability  incentives  and  penalties.  The  precise  nature  and  the  percent 
effect  on  the  fee  can  vary  considerably  depending  on  the  nature  and 
operational  use  of  the  equipment  being  procured.  The  exact  fees  are 
determined  during  pre -contractual  negotiations.  Bidders  are,  however, 
encouraged  to  respond  or  to  suggest  alternate  incentive/penalties  arrange¬ 
ments.  There  is  Air  Force  interest  in  varying  arrangements  in  order  that 
sufficient  experience  may  be  gained  and  eventually,  realistic  relation¬ 
ships  established  that  will  not  unduly  penalize  either  the  military  cus¬ 
tomer  or  the  industrial  concern  developing  new  electronic  systems. 
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Figure  3-  Progressive  Relationship  for  Reliability  Incentives/Penalties 


SECTION  IV 


RELIABILITY  ECONOMICS  FACTORS 


Technical  progress  in  the  state-of-the-art  for  reliability 
has  always  been  an  important  factor  in  technical  proposal  evaluations. 
Experience  with  many  conthacts  demonstrated  that  the  following  aspects 
of  reliability  are  equally  important. 

Effect  of  Reliability  Upon  Maintenance  Costs 


Requirements  for  high  reliability  invariably  result  in  increased 
contract  costs.  The  Air  Force  is  interested  in  total  costs,  not  just 
the  initial  purchase  price  of  the  equipment.  Reliability  has  a  dramatic 
impact  upon  spares,  manuals,  test  equipment,  checkout  devices,  calibra¬ 
tion,  and  all  logistic  actions.  Technical  proposals  should  not  justify 
reliability  costs  on  increased  performance  alone.  Reliability  costs 
should  specify  the  impact  and  the  potential  savings  in  the  area  of 
maintenance . 

Mr.  E.  J.  Nucci  (DDR&E)  in  the  September  I963  IEEE  Trans¬ 
actions  on  Reliability  paper,  titled  "Concepts  and  Research  Needs  of 
Reliability  in  Military  Systems,"  released  the  following  information: 

a.  Current  DOD  maintenance  budget  is  more  than  $10,000,000,000 

per  year. 

b.  960,000  military  and  civilian  personnel  utilized  for 
maintenance  alone. 

c.  Contractor  maintenance  of  military  equipment  will  decrease. 

d.  Military  maintenance  costs  are  expected  to  increase. 

Ten  billion  dollars  per  year  is  approximately  20%  of  the 
National  Defense  Budget.  Prorated,  this  is  more  than  1,000,000  tax 
dollars  for  each  and  every  hour  of  the  year,  just  for  the  maintenance 
costs.  Predicted  reliability  has  become  one  of  the  important  tools 
used  by  evaluators  to  assess  the  overall  cost  including  operation  and 
maintenance . 

Technical  proposals  should  use  various  approaches  to  indicate 
the  potential  contractor's  concern  with  the  military  maintenance  problem. 
Figure  Ii.  shows  the  effect  of  component  parts  failures  on  the  ultimate 
logistic  costs.  The  percentage  of  failures  is  directly  proportional  to 
replacement  costs.  The  average  unit  replacement  cost  of  $20  per  failure 
was  developed  as  the  result  of  a  study  by  the  RCA  Service  Co.  for  the 
Rome  Air  Development  Center,  and  was  detailed  in  a  paper  by  Messrs  R.  L. 
McLaughline  and  H.  D.  Voegtlen,  "Ground  Electronic  Equipment  Support 
Cost  vs  Reliability  and  Maintainability."  The  costs  include  material, 
manpower,  transportation  and  administrative  costs,  but  exclude  such 
additional  costs  as  loss  of  product,  storage  facilities,  depots,  pipe 
lines,  spare  parts  stockage,  etc.  The  effect  of  selection  of  parts,  in 
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percentage  failure  per  1000  hours,  has  been  charted  for  a  high  density 
ground  electronic  system.  The  direct  relationship  between  failures  and 
replacement  costs  is  shown.  Using  the  basic  cost  factor  of  $20.,  the 
table  of  costs  is  based  upon  the  four  failure  rates  listed  as  basic 
requirements  by  the  "Darnell  Report"  Parts  Specification  Management  for 
Reliability  (PSMR). 
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From  the  elementary  figure  in  Figure  i;,  it  can  be  readily  seen 
how  much  can  be  saved  over  a  period  of  ^0,000  hours  (the  normal  life  of 
ground  electronic  equipment)  by  selecting  the  next  highest  reliability 
level  of  parts.  Taking  as  an  example  the  reliability  levels  of  0.01^  - 
0.001^  failures  per  thousand  hours,  a  saving  of  U5}000,000  is  accrued 
over  the  50,000  hour  period  by  choosing  the  o.ool^  failures  per  thou¬ 
sand  hours  of  operation.  These  savings  applied  during  the  design  and 
development  phases  would  result  in  many  other  savings,  in  addition  to 
any  improved  end  product.  These  additional  savings  would  include  such 
items  as  reduction  in  manpower  and  training  requirements,  less  redun¬ 
dancy,  fewer  number  of  items  in  the  logistics  supply  lines,  less  trans¬ 
portation  and  administrative  costs,  less  storage  facilities,  and  less 
depot  support  requirements.  Although  these  figures  are  approximate  and 
cover  a  small  area  of  the  economy  of  reliability,  it  is  readily  apparent 
that  many  benefits  and  economies  may  be  achieved  through  design  for  relia¬ 
bility.  Bidders  are  encouraged  to  use  similar  approaches,  particularly 
when  it  is  the  considered  opinion  of  the  prospective  contractor  that  it 
is  feasible  within  the  state-of-the-art  to  exceed  the  reliability  re¬ 
quirements  being  specified  in  the  Request  for  Proposal. 

Reliability  as  a  Cost  Factor 


AFR  80-5,  "Reliability  Program  for  Systems,  Subsystems,  and 
Equipment,"  states  "Proposals  for  increasing  reliability  efforts  on 
programs  in  existence  will  be  considered  on  the  basis  of  the  net  effect 
on  the  overall  Air  Force  capability  and  economy,  including  such  factors 
as  spare  parts  requirements,  maintenance  workloads,  engineering  changes, 
operating  costs,  and  the  effect  on  the  system  concerned."  Figure  5 
illustrates  this  point,  and  shows  the  relationship  between  the  relia¬ 
bility  of  complex  electronic  systems  and  total  systems  cost.  Figure  5 
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is  an  adaption  of  the  AGPIEE  Mission  Cost  vs  Reliability  model.  As  the 
design-for-reliability  is  emphasized,  a  proportional  reduction  in  the 
Support  Costs  is  reflected-  There  is,  however,  an  optimum  point  beyond 
which  additional  design  effort  on  reliability  may  not  be  warranted.  At 
this  optimum  point,  the  design-for-maintenance  becomes  an  important  con¬ 
tributor  . 


Figure  5,  page  2U,  indicates  how  an  initial  expenditure  in 
time  and  fluids,  to  improve  current  reliability  levels  (shaded  area), 
will  reduce  total  costs.  The  curves  are  typical  of  those  that  could  be 
developed  by  bidders  and  used  to  establish  reliability  requirements  de¬ 
fining  a  system  of  maximum  value  to  the  customer. 


23 


TOTAL  COSTS  —  DOLLARS 


Figure  5-  Relationship  Between  Reliability  and  Costs 
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SECTION  V 


APPROACHES  FOR  INCREASED  RELIABILITY 


Basic  Reliability  Approach 


The  reliability  of  electronic  equipment  largely  depends  upon 
the  follovdng  fundamental  factors: 

a.  The  inherent  design  of  the  circuitry  and  the  technique 
of  packaging  of  the  hardware- 

b.  The  reliability  of  the  components  and  materials  used. 

c.  The  workmanship,  assembly,  and  control  techniques  during 
manufacture • 


d.  The  operational  use  and  application  of  the  equipment. 

e.  The  maintenance  techniques  used  after  installation  of 
equipment  in  its  operational  status. 


Past  experience  suggests  that  there  is  another  facet  which 
affects  reliability;  as  the  complexity  of  the  equipment  is  substantially 
increased,  the  reliability  levels  depend  primarily  on  complexity.  As 
complexity  increases,  the  reliability  level  decreases. 

Figure  6  is  a  chart  developed  by  the  Rome  Air  Development  Cen¬ 
ter  which  describes  RADC's  experience  with  ground  radar  equipments. 
Figure  6  shows  historically  and  statistically  that  as  the  number  of 
active  elements  increased,  the  reliability-probability  of  operating  for 
a  2\x  hour  period  decreased.  At  this  point  in  time,  reliability  in 
MTBF  appears  to  have  leveled  off,  but  Air  Force  radar  sets  are  becoming 
more  complex. 

However,  this  is  true  only  for  series  type  factions.  In 
actual  practice,  one  ground  electronic  system  has  been  made  deliberately 
complex  to  achieve  the  necessary  reliability. 


Increased  Reliability  Through  Redundancy 

Ground  electronic  equipment  traditionally  has  rarely  had  the 
physical  space  limitations  that  are  inherent  in  aeronautical,  missile 
or  space  systems.  Generally,  ground  electronic  equipment  has  had  lit¬ 
erally  acres  of  land  on  which  the  equipment  could  be  installed.  Physical 
weight  has  only  occasionally  been  a  primary  consideration.  Power  sources 
were  also  a  secondary  consideration,  and  never  a  serious  limitation,  be¬ 
cause  a  few  additional  kilowatts  of  power  could  always  be  secured  by 
introducing  additional  or  increased  capacity  motor  generators,  etc.  The 
result  has  been  ground  electronic  equipment  that  is  generally  and  per- 
missably  bulky,  heavy,  and  consumes  sizeable  quantities  of  power.  Addit¬ 
ionally  the  maintenance  and  transportability  aspects  are  increased. 

As  ground  electronic  eqxiipment  grew  more  complex,  designers 
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Reliability  Figure  6.  Radar  Complexity  vs.  Reliability 

(Not  including  redundancy) 


used  a  common  technique,  equipment  redundancy,  to  gain  increased  relia¬ 
bility.  For  example,  large  scale  computers  were  duplicated  to  insure 
reliability  and  operational  availability.  Radar  klystrons  were  paralleled 
to  provide  switch-over  to  the  stand-by  unit  in  the  event  of  failure.  In 
some  cases,  complete  receivers  and  transmitters  are  maintained  in  stand¬ 
by  readiness,  to  permit  and  assure  re liability/ availability.  All  these 
techniques  were  permitted  with  the  reluctant  approval  of  the  Air  Force, 
for  expediency  and  lack  of  other  approaches.  The  result,  however,  has 
been  an  inordinate  and  astronomical  increase  in  the  support,  logistics 
and  maintenance  aspects.  More  than  30  percent  of  the  Air  Force  budget 
is  for  maintenance  of  today's  systems,  and  that  maintenance  of  electronic 
equipment  ranges  between  sixty  and  one  thousand  times  the  initial  costs. 
Redundancy  techniques  of  the  type  described  are  therefore  no  longer  en¬ 
couraged. 

Reliability  Through  Increased  Complexity. 

One  redundancy  technique  which  has  been  generally  accepted  is 
that  used  by  Phased  Array  Radars.  The  basic  concept  in  Phased  Array 
permits  the  use  of  low  power,  identical  transmitters  by  the  introduction 
of  phase  shift  to  result  in  an  electronically  steerable  high-power 
antenna.  Basically,  phased  array,  as  used  in  ground  electronics,  consists 
of  some  5000  identical  low-power  transmitters  and  is  essentially  a  case 
of  "natural  redundancy". 

Phased  Array  virtually  eliminates  major  failures  in  radar  re¬ 
ceivers  and  radar  transmitters.  Additionally,  the  failures  associated 
with  the  antenna  dish,  the  antenna  feeds,  the  pedestals  and  the  servos 
are  almost  eliminated.  It  is  possible  to  have  10^  of  the  ^000  trans¬ 
mitters  defective  and  yet  maintain  negligible  loss  of  power  and  effi¬ 
ciency.  Basically,  the  "series"  operation  problem  has  been  eliminated. 
Phased  Array  is  discussed  here  to  indicate  that  although  brute-type 
redimdancy  is  not  generally  favored,  there  are  techniques  such  as  Phased 
Array  that  are  acceptable,  and  that  have  reached  satisfactory  levels  of 
reliability,  although  parts  complexity  is  extremely  high. 

Series -systems  such  as  Telstar  and  MLnuteman  have  been  proven 
reliable;  therefore,  when  prospective  contractors  use  brute  redundancy 
as  the  primary  approach  to  the  required  operational  reliability,  the 
proposals  will  receive  particularly  close  scrutiny.  It  is  also  possible 
that  lower  evaluation  weights  will  be  assigned  to  such  proposals. 
Sophisticated  redundancy  approaches  such  as  the  Phased  Array  radar  are 
encouraged  and  should  be  introduced  during  the  Technical  Proposal  eval¬ 
uation  stage. 
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SECTION  VI 


CONCLUSION 


The  procuring  agency  should  initially  define  the  job  to  be 
undertaken  in  great  enough  detail  so  that  the  prospective  contractor  may 
understand  what  the  Air  Force  wants.  It  then  becomes  the  responsibility 
of  the  bidder  to  simply  and  clearly  describe  what  he  will  deliver.  It 
is  not  sufficient  for  the  bidder  to  merely  discuss  or  develop  each  of  the 
reliability  requirements  listed  or  referred  to  in  the  Request  for  Pro¬ 
posal.  It  is  necessary  that  the  bidder  develop,  emphasize,  and  prove 
to  the  military  customer  that  he,  the  contractor,  will  have  a  dynamic 
reliability  program  that  will  be  tailor-made  to  a  particular  program. 
Further,  reliability  cannot  be  treated  as  a  separate  and  peculiar  science; 
it  has  an  impact  upon  many  other  important  areas,  and  it  is  important 
that  the  interactions  and  tradeoffs  with  maintainability,  quality  assur¬ 
ance,  logistics  and  maintenance  be  fully  developed  in  the  Technical  Pro¬ 
posals  submitted. 
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